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ABSTRACT 

A sound performance evaluation system uses a 
multitude of assessment methods to gather information and data about 
the performance of an individual. Four models are described that have 
been used with multiple assessment results to make decisions about 
performance. In a compensatory model, weak performance on one measure 
or attribute can be traded off against strong performance on others 
when calculating a score. The conjunctive model requires that the 
individual attain a minimum level of performance (e.g. cutoff or 
passing score) on each of the assessments. A combined model uses 
features of both the compensatory and conjunctive models, and is used 
when decisions are made at two or more decision-making levels. The 
first three models require some minimum level of performance by the 
individual on all assessments, but the disjunctive model requires 
only that the individual have an acceptable level of performance on 
one of the assessments to demonstrate satisfactory or minimally, 
acceptable performance. Examples are provided for each of these 
models and are summarized in Table 1. (Contains 7 references.) 
(SLD) 
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Decision Making Models for Using Multiple Assessment Results 

Patricia H. Wheeler, M.B.A., Ph.D. 
EREAPA Associates 
Liverriiore, California 



A sound performance evaluation system uses a multitude of assessment methods to 
gather information and data about the performance of an individual (Shulman, 1988; . 
Wheeler, 1993). Each of these assessments yields a result and, in some cases, multiple 
results or scores. To make decisions about education, career and personnel actions (e.g., 
passing a course, admission to a college, licensure, hiring, reten'ion/dismissal, 
promotion), data must be combined in some appropriate maimer and subjected to 
prespecified decision rules. 

Several approaches have been identified for using multiple assessment results to make 
decisions (Mehrens, 1990; Scriven, 1991). Four of these are described below; examples 
are provided for each and summarized in Table 1. 

Compensatory Model 

In a compensatory model for using multiple assessment results, weak performance on one 
measure or attribute can be traded off against strong performance on others when coming 
to a decision or calculating a total score. For instance, a cook might make excellent 
salads and sandwiches, but does only a fair job of making soups. The owner might be 
willing to hire or retain such an individual in a restaurant, even though further training on 
preparing soups would be required, or another employee hired to make soups. A student 
might be able to pass a course, even though he/she failed an important unit test or the 
final examination, if results on other tests during the course were high enough to offset 
this poor performance. ! 

Usually compensatory models have a minimum required level of performance whereby 
"overscores" in one area can offset "underscores" in another area. That is, performance 
below some absolute minimum level of performance cannot be offset by high 
performance on another assessment (e.g., a median rating of 7.5 on an assessment is 
passing; a 6.5-7.4 on this assessment can be offset by a 9.0 or higher on another 
assessment; anyone with a median rating below 6.5 on this assessment does not pass). 
An individual might be allowed to offset one "unacceptable" result with one "excellent" 
result, but cannot offset two "unacceptable" levels with two, or even three, "excellent" 
results. An individual might be allowed to receive an "unacceptable" level in some 
attributes, but not in other attributes (regarded as critical). These are examples of issues 
that must be covered in the policy's decision rules for using multiple assessment results. 



Conjunctive Model 

The conjunctive model for using multiple results requires that the individual attain a 
minimum level of performance (e.g., a cutoff or passing score) on each of the 
assessments. This approach is suitable when all attributes being assessed are critical for 
minimally acceptable performance. For example, a paramedic should demonstrate a 
minimum level of performance on several tasks (e.g., controlling bleeding, rescue 
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breathing, cardio-pulmonary resuscitation, moving an injured person). A paramedic who 
could only do three of these at a minimally acceptable level and could only learn the 
fourth with time and experience is not the most desirable person at the scene of an 
emergency. 

In a small business, a conjunctive model approach can be important. Reliance on one 
person to manage an office or a shop if he/she could not satisfactorily perform all 
necessary tasks would be irresponsible. For example, a carpet salesperson must be able 
to explain the advantages and disadvantages of different types of carpets, be familiar with 
the offerings available through the store (either on hand or with special order), provide 
information about available installation services, operate the cash register, and process 
credit card purchases. If rile salesperson staffing a store could not perform such essential 
tasks at a minimally acceptable level, he/she might cause financial loss to the owner. 

A conjunctive model is typically used for driver tests. To obtain a license, a person must 
pass a written test of knowledge, a vision examination, and a road test. We would not 
want to issue driving licenses to persons who passed two of the three tests, even if they 
barely failed one of them and did quite well on the other two. 



Combined Model 

A combined model uses features of both the compensatory and conjunctive models. This 
model is used when decisions are made at two or more decision-making levels. For 
example, students must complete a series of assessments to pass certain courses. Usually, 
at the course level, teachers use a compensatory model. That is, a student can pass a 
course even if he/she fails one or more tests or assignments. However, to graduate, the 
school may use a conjunctive model where each of several requirements must be met. 
Such requirements might be passing a certain number of courses in various subject areas 
with at least a "C," writing a satisfactory senior thesis, and passing all three tests in an 
academic skills battery. 



Disjunctive Model 

The first three models require some minimum level of performance by the individual on 
all assessments. This is not the case with the disjunctive model. An individual needs 
only to have an acceptable level of performance on one of the assessments used to 
demonstrate "satisfactory" or "minimally acceptable" performance. 

This model is defensible when there are several ways to demonstrate satisfactory 
performance or multiple assessments of the same performance attribute. The disjunctive 
model may also be appropriate in cases where ieiakes are permitted; here, users may 
consider only the highest score, or a typical/average score, or the most recent score, and 
drop the other scores on that same assessment from consideration. 

The disjunctive model is not appropriate if all domains are essential for satisfactory 
knowledge of a subject or performance in a job. A student would not pass an American 
history course if he/she only passed the test on the Civil War era and failed the tests on 
the other periods of American history. 

The disjunctive model is probably the most efficient ot the models for the number of 
assessments given. If an individual demonstrates satisfactory performance in an area on 
one assessment, then that individual need not be further assessed in that area. For 
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example, if a teacher demonstrates satisfactory written communication skills by preparing 
an informational memorandum for other teachers at the school, then he/she will not have 
to complete other measures of written communication, such as a letter to parents on their 
child's progress during the past two months, or a short essay summarizing the content of 
a proposed text book. A college student can demonstrate fulfillment of a course 
requirement by taking and passing that course, or by passing a test, or by completion of 
an independent study program, or by completing a series of related courses. 



Table i. 

Examples for Each Model for Four Assessments 



Model Assessment: Decision 

A B C D 



Compensatory 

Person 1 Pass High Pass Barely Fail Pass Pass 

(B offsets C) 

Person! Pass Barely Fail Pass Pass Fail 

(B not offset) 



Conjunctive 

Person 3 Pass Pass Fail Pass Fail 

(must pass C) 

Person 4 Pass Pass Pass Pass Pass 

(passed all) 



Disjunctive 

Person 5 Fail Pass Fail Fail Pass 

(must pass 
only one) 

Person 6 Pass Pass Pass Fail Pass 

(only had to 
pass one) 
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Summary 

There must be a sound rationale for the selection of a model and the specification of a 
formula or rule for use in making decisions based on multiple assessment results. 
Thorndike and Hagen (1955) emphasize the importance of basing procedures for 
combining multiple results on empirical evidence (e.g., regression analysis) rather than 
intuitive judgments, although this is possible only if sufficient empirical data are 
available. 

Standards for satisfactory (or passing) performance levels for each assessment used in the 
evaluation process must be set in an. appropriate manner (see Livingston and Zieky, 
1982). Thoughtful deliberation must be given to the tradeoffs of fairness to different 
parties. Giving the benefit of the doubt to the individual versus the needs of others and of 
protecting clients from individuals who should not be licensed or working in certain 
situations should be considered carefully. 

Whatever decision making model is adopted, the policy must state the rationale for 
selecting it. The procedures for implementing the model and the process of using 
multiple results for decision making must be provided, with adequate lead time, to all 
involved, including the individuals being assessed. Decision makers must be carefully 
trained and procedures should be in place for ongoing monitoring to ensure that the 
models are implemented in a sound, fair, and accurate manner. 

Validity issues should also be addressed in the policy. The following questions are 
examples of matters concerned with validity. Are those students who have been 
admitted to the college able to succeed academically? Are new employees able to 
perform their jobs in a minimally acceptable manner? Are students who passed a course 
or a grade level able to do academic work in the next course or grade level? Do 
individuals who are licensed to practice a career perform in accordance with the 
regulations and guidelines for that field? 

The possibilities for retakes of each assessment used and an appeals process must be 
formulated and in place so that individuals can request another assessment administration 
or can challenge decisions based on the use of these models. Exceptions to these 
procedures and the decision i ules should be made with care, for once an exception is 
made, there will likely be pressure to make more exceptions. A review of the 
appropriateness of the model and the decision rules based on it should be made at least 
once every three years, and changes made if needed. 

Given the plethora of data available for assessing an individual's performance, it is 
critical that all information be used in a manner that will lead to sound decisions, both for 
the sake of the individual and for their clients, their employers, and their community. 
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